JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 1
Attitudinal effects of data visualizations and
illustrations in data stories
Manuela Garret
´
on, Francesca Morini, Pablo Celhay, Marian D
¨
ork and Denis Parra
Abstract—Journalism has become more data-driven and inherently visual in recent years. Photographs, illustrations, infographics,
data visualizations, and general images help convey complex topics to a wide audience. The way that visual artifacts influence how
readers form an opinion beyond the text is an important issue to research, but there are few works about this topic. In this context, we
research the persuasive, emotional and memorable dimensions of data visualizations and illustrations in journalistic storytelling for
long-form articles. We conducted a user study and compared the effects which data visualizations and illustrations have on changing
attitude towards a presented topic. While visual representations are usually studied along one dimension, in this experimental study,
we explore the effects on readers’ attitudes along three: persuasion, emotion, and information retention. By comparing different
versions of the same article, we observe how attitudes differ based on the visual stimuli present, and how they are perceived when
combined. Results indicate that the narrative using only data visualization elicits a stronger emotional impact than illustration-only
visual support, as well as a significant change in the initial attitude about the topic. Our findings contribute to a growing body of
literature on how visual artifacts may be used to inform and influence public opinion and debate. We present ideas for future work to
generalize the results beyond the domain studied, the water crisis.
Index Terms—data stories, attitude change, emotions, quantitative and qualitative evaluation
✦
1 INTRODUCTION
J
OURNALISM is experiencing profound transformations
that are related to a range of socio-technological devel-
opments. In particular, the maturation of the web funda-
mentally changed the ways in which content is created,
distributed, and consumed. At first, traditional news media
responded reluctantly [1], but over the last decade on-
line newsrooms have experimented with more innovative
forms of storytelling. Some of these innovations involve
integrating various visual elements into interactive article
formats, including data visualizations, [2], [3], [4], [5], [6].
Visual representations embedded into online articles are
often used to help convey complex issues of public concern,
and therefore influence the way readers form an opinion
about it. In recent years, interest in visualization as part
of data stories has grown beyond academic visualization
research into professional fields such as data journalism [4],
[7]. While there are a range of assumptions at play about
the rhetorical power of visualizations and illustrations in
data stories, so far there is limited empirical evidence about
their attitudinal effects.
In a data journalism context, data visualization usually
operates in a space with different forms of visual represen-
tation such as illustration, photography, and videos. We are
especially interested in understanding illustration in this
• M. Garreton - Department of Computer Sciences, Pontificia Universidad
Cat´olica de Chile, E-mail: manuela.garreton@uc.cl
• F. Morini - Urban Complexity Lab, University of Applied Sciences
Potsdam.
• P. Celhay - School of Government, Pontificia Universidad Cat´olica de Chile
• M. D¨ork - Urban Complexity Lab, University of Applied Sciences Pots-
dam
• D. Parra -Department of Computer Sciences, Pontificia Universidad
Cat´olica de Chile,
Manuscript received April 19, 2005; revised August 26, 2015.
study, since we see it as part of many related topics that
have been studied within the data visualization research
community; efficacy of embellishments [8], [9], infographics
[10], anthropographics [11], and data comics [12]. Data
visualization and illustration have been studied within these
in an integrated way, and as such have been understood as
communicative visualization techniques that are typically
coupled. For example, the appropriate integration of visual
embellishment in data representation may help viewers
grasp key concepts more effectively [9]. In contrast, in our
study we try to separate the effect of data visualization from
illustration, trying to compare how these distinct visual
representations operate individually and in combination
with each other within an article. We are interested to find
out how readers respond to these two related types of
visual representations in the context of a particular story
pertaining to a current issue of public concern. In short,
we ask: How do the attitudinal effects of data visualization,
illustration, and their combination in data stories compare?
The effect of visual representations embedded in jour-
nalistic articles has been addressed mainly by research in
journalism and framing studies, where the notion of visual
framing focuses on the characteristics of images and their
respective effects on readers and their sensemaking [13].
Most framing studies on photographs and illustrations [14],
[15], [16], [17], [18] suggest that images hold a stronger
framing effect than text alone. They can directly influence
behavior and increase the likelihood of acting in response to
the presented stimuli [18], [19]. Photographs especially hold
a strong rhetorical power [15], [16], capable of permanently
modifying readers’ agenda, and impacting the perceived
importance of a given issue [14]. While framing theory
has been used to inspect rhetorical techniques in narrative
visualizations [20], there is little empirical work on the
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 2
difference in attitudinal effect between data visualization,
illustration, and their combination in data stories.
We define attitudinal effects as a change in attitudes
towards a specific topic. We focus our research on attitude
changes as they can be considered a central effect of data
stories. So far, there is limited understanding of the ways
different types of visual representations can alter attitudes.
The term attitude is regarded as how a person evaluates
issues, objects, or other people [21]. Changes in attitudes
can also be explained by the Elaboration Likelihood Model
of persuasion [22], where persuasion plays an important
role together with emotions [23]. On the other hand, for the
change of attitude to be lasting, it must be accompanied
by the formation of a satisfactory memory (information re-
tention) [24], [25]. Therefore, as we seek to shed light on the
attitudinal effects of visual representation in data stories, we
will measure persuasion, emotion and information retention
in an empirical study. We also consider it necessary to study
these dimensions in combination, and not in isolation as
has been done in previous visualization studies on emotions
[e.g., 26, 27], persuasiveness [28], and memorability [e.g., 29,
30].
Furthermore, despite constituting essential components
of articles, visual representations—in particular, information
visualizations—are usually analyzed independently of their
context, i.e., separate from the story text. Visualization schol-
ars tend to construct laboratory settings to study the impact
of journalistic data visualization on readers [e.g., 8], [31],
possibly as an attempt to contribute clear methodologies
and results. However, we seek to understand the impact
of visual representations in practice by studying them in-
tegrated with text. In this way, we try to maintain the
ecological validity of the research.
We also contribute to an improved understanding about
the attitudinal effects of visual representations embedded
in journalistic storytelling. Besides the effect on attitudes
immediately after exposure, we are interested in the sta-
bility of changes, i.e., the change lasting over time. For this
purpose, we present a web-based study of an instrumented
data story with responses collected immediately before and
after reading the story, and after a week. By comparing three
different versions of the article (illustration-only, datavis,
and combined) we observe how attitudes change based on
which visual stimuli are present and how they are perceived
when combined. We compare the effects of visualizations
and illustrations on readers’ attitudes along three crucial
dimensions: persuasion, emotion, and information reten-
tion. Our findings suggest that data-driven articles, where
information is visually rendered by charts and maps, have
a higher impact on readers’ attitude compared to articles
containing illustrations alone.
2 B ACKGROUND
In this section, we first address the types of visual repre-
sentations which are the focus of our research, how these
have been studied previously and how our approach differs.
We then review how persuasion, emotions and information
retention play an important role in attitude changes.
2.1 Visual representations
The communicative and rhetorical salience of visual repre-
sentations have been studied across a variety of domains,
including communication and journalism studies [e.g., 32]
and information visualization [e.g., 20]. While journalists
often integrate a variety of visual artifacts in their articles,
previous studies tend to center on one type, for example,
photographs, and their effects on readers [14], [15], [16], [17],
[18].
In this study, we compare the effect of two common
types of visual representations—visualizations and illustra-
tions—in the context of journalistic storytelling. We chose
them as two distinct visual representations that tend to
be used in combination in infographics and data stories,
and seem to serve distinct communicative functions. While
data visualizations represent abstract data [33], illustrations
depict concrete objects and scenes. Arguably, the former
suggests a kind of evidence potentially based on official
or scientific sources and which people see as impartial,
therefore trusting them [34], [35], while the latter com-
municates a topic in a more figurative way. Moreover, it
plays a role in several studies on visualization that, through
different approaches, have given an account of their un-
derlying communicative techniques and their relationship
with visualization. Illustration has a space for embellishing
visualizations, and therefore its uses have been debated.
Some advocate removing them completely [e.g., 36] but
other empirical studies show that they improve information
retention [8], [9]. The visual embellishment or uniqueness
of a visualization can positively impact its memorability [8],
[31]. Comparing different chart types, visualizations with
strong aesthetic connotations are more likely to be retained
and effectively recalled after a long period of time. Specific
types of illustration have also been studied. For example,
the use of pictographs in visualization can help people
remember information during demanding tasks and entice
them to inspect a visualization more closely [37]. However,
another study observed that the use of anthropomorphisms
in visualization has similar effects in eliciting empathy and
prosocial behavior compared to standard charts [11]. Our
approach differs from prior studies, as it seeks to first iden-
tify the effects of illustration and visualization separately
and then in combination.
2.2 Attitudinal change
According to Petty [21]: “The term attitude is used to refer to
a person’s overall evaluation of people (including oneself),
objects, and issues”. They can be evaluated either positively
or negatively. Consequently, attitude change means that the
previous evaluations of an individual are modified.
One attitude changing method is through persuasion:
the act of providing additional, new or contradictory in-
formation to modify individuals’ evaluation of the topic
at hand [21]. Consequently, persuasion can be understood
as a quality of information which makes change in initial
attitudes possible. While it can be achieved through analyt-
ical reasoning or emotional appeal [38], persuasion is often
described as the process to influence other humans to adopt
new attitudes, opinions, beliefs, or to change their behavior
altogether [39]. In social psychology, persuasion is believed
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 3
Fig. 1. Two distinct visual representations embedded in a data story about the water crisis in Chile: an illustration of an almost dried up riverbed
(left) and a visualization of varying rainfall in a territory (right)
to play a key role in how people make choices and perform
actions [21]. The foundations of the most widely accepted
theoretical model of persuasion is ELM (Elaboration Like-
lihood Model), which relies on personal relevance ascribed
towards a topic [22]. The role of visualization in persuading
viewers has been addressed in different domains, including
environmental governance [40], climate change [41], [42]
and human rights [43]. Pandey et al. [28] experimentally
studied the persuasive power of visualization, and identi-
fied that charts rather than tables lead to higher persuasion
when participants do not possess a strong initial attitude
about the topic. However, when they have a strong initial
attitude they are less likely to be persuaded.
Attitudes are also strongly influenced by emotions. Ac-
cording to ELM, emotions can modify attitudes depending
on the total elaboration level of the message [23]. When
individuals’ ability for elaboration is low, emotions have a
high impact on attitude. However, even when the ability for
elaboration is high, emotions can still impact persuasion by
using the feelings which people have as argument, biasing
the overall cognitive process. This happens because emo-
tional arguments are formulated to cause people to like or
dislike their own thoughts (affective validation) or to feel
more confident or doubtful (cognitive validation) [23]. Gen-
erally, emotions are understood as requiring less elaboration
effort. When subjects do not have strong initial beliefs on a
particular topic, they elaborate stimuli peripherally [44]. In
this regard, images are proven to have a higher emotional
impact than text [45]. Readers connect more quickly with vi-
suals and tend to be more persuaded [18]. In general, stories
enriched by visual artifacts, specifically photographs, tend
to foster a stronger emotional response in readers and lead
them to permanently change their attitude [19]. The types of
images influence the framing effects for the same topic. The
emotional valence of certain stimuli impact the way readers
retain information and form opinions [17]. Emotions play a
fundamental role in making sense of numeric and complex
data as well [26]. The temporal and geographic proximity
of a topic and the personal, educational and political back-
ground of a viewer can greatly influence the emotive impact
of a visualization [27], [46]. Recently, approaches that value
the inherent affective aspects of data visualizations have
emerged, devising guidelines to explicitly foster emotions
in readers and render critical topics more actionable [47],
[48].
A central aspect of attitudinal change is its longevity,
i.e., information retention. Attitudes are represented in
memory as summary evaluations associated with the at-
titude object [49]. People tend to have better memory for
negative than positive stimuli [50]. In the news context,
visual artifacts improve readers’ ability to recall content
[51]. Dual-mode presentation—when text is associated with
images—performs better than single-mode, where only text
is present [52]. The presence of images can distort memories.
If incorrect pictures are associated with text, people are more
likely to remember images and fabricate the content of the
accompanying text [53].There are a variety of studies con-
cerning data visualizations and memory. Most of them focus
on which elements are relevant for a visualization to be
memorable: graphical and textual elements such as layouts,
titles, and legends foster memory [29]. In particular, titles
influenced the recalled main messages even though they can
be misaligned with the message of the visualization [35].
In comparison to interactive visualizations, author-driven
narratives may be more beneficial for data comprehension,
but the difference in long-term information recall is unclear
[30].
Another concept related to attitudinal changes and stud-
ied and studied in the visualization community is belief
updating. Recent studies consider that prior beliefs held
by people affect data visualization comprehension. These
works apply Bayesian models to evaluate and explain belief
updating from visualizations [54], [55], [56]. Our work, by
contrast with those mentioned in this section, seeks to study
the dimensions of persuasion, emotion, and information
retention in combination, rather than in isolation.
3 STUDY DESIGN
To better understand the effects of visual representations
on readers’ attitudes, we are interested in three central at-
titudinal dimensions: persuasion, emotion, and information
retention. We created realistic journalistic content that par-
ticipants would—and did—consume in their daily lives. We
prepared three versions of an article that included different
types of visual representations, and ran several evaluation
phases to collect quantitative and qualitative data. This en-
abled us to compare the role of visualization and illustration
in data stories, while exploring these visual representations’
effects on readers. In particular, we are interested in the
immediate and short-term change (or stability) of attitudes,
emotions, and correct response rates.
3.1 Article design & visual artifacts
We wrote, designed and developed one article that encapsu-
lates different types of visual representations. This allowed
us to ensure that participants had not read it beforehand.
We also had precise control of the article characteristics that
are relevant for the study. These include length, equivalent
conditions in terms of content and visual representations,
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 4
Fig. 2. We implemented three versions of the same data story on the water crisis in Chile: Participants read one of three versions; version A with
illustrations only, version B with visualizations only, and version AB with both. The narrative of the data story was organized in 4 sections (A dry
future, Mining thread, An unequal flow, A leaking system). In the figure we present screenshots of each section. showing how visual representations
are related with text in each condition (A, B and AB). The complete stories can be seen in the supplementary material.
number of illustrations versus visualizations, simplicity of
the visualizations, and other factors. The article is a long
read introducing and discussing the Chilean water crisis.
The topic was initially chosen because of its limited cov-
erage in traditional media, despite the fact that it will
have a major impact on citizens’ daily lives over the next
decade, and because when it does get discussed, it is often
presented as a problem caused by climate change alone.
These two characteristics left us room to explore the topic
further, and to devise an engaging way of presenting the
content. To better understand the topic, we read scientific
studies and public reports from various sources (e.g., NGOs,
governmental institutions and international organizations).
These revealed different reasons for causality that are not
clearly delineated in the public debate. We structured our
article to revolve around four selected arguments: water
management inefficiency, mining as a disrupting human
activity, social inequality among the Chilean population,
and governmental responsibility in shortcomings. Our main
goal was to build a convincing narrative thread, to persuade
readers of how different problems contribute concurrently
to the Chilean water crisis. We intended to convey the
complexity of the problem at hand, instead of focusing on
one single argument.
To develop a strong and convincing narrative thread,
we organized a workshop with experts in hydrology, law,
geography, and economics. The insights and ideas generated
during this workshop informed our story design and devel-
opment, including the visualizations and illustrations. The
first draft of the article was written to provide geographical
and social context for the water scarcity problem in Chile.
Later on, we hired a journalist, who integrated new sources
and edited a version of the text ensuring a journalistic
and appropriate language for a general audience. The final
outcome is a long-read article (2500 words approximately)
divided into four sections, plus introduction and conclu-
sion. Each section addresses a different dimension of the
water problem: droughts, mining, social inequality, and in-
stitutional mismanagement. Illustrations and visualizations
were designed to match and complement the individual
sections’ content, following a fairly simple style.
• Illustrations: We included four illustrations in the article
along with the header, which is a combination of the
individual designs. We invited an illustrator, who in an
iterative process proposed an illustration for each section
of the article. The aim was to develop images that visually
summarized and represented the content of each section.
Some illustrations depict concrete elements (e.g., the im-
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 5
Fig. 3. The study is structured into three main observational phases: before reading the article (t0), just after reading the article (t1), and a
week after reading the article (t2). N corresponds to the number of participants for each phase (t0,t1, t2) and in each condition (A, B or AB).
Subsequent analysis is performed by observing the difference among the results obtained between t1 and t0 (t1-t0) and then the difference
between t2 and t0 (t2-t0)
pact of droughts and human activities on the landscape),
while others present abstract concepts (e.g., unequal water
distribution, institutional power and shortcomings).
• Visualizations: In total the article includes 12
visualizations—3 maps, 3 infographics, and 6 charts—
most of them static. Four of them use readers’ scroll
position to change their status and trigger animations.
For each section, we designed one main visualization.
In the introduction and conclusion we added secondary
graphs (see Figure 2). They were designed by one of the
authors, who is a data visualization professional. We
aimed to make these visualizations as realistic as possible,
so we drew on current trends in major newspapers as a
reference. We conducted a survey of data stories about
water issues where the use of maps and infographics is
persistent [e.g 57,58,59].
We follow essential characteristics of data stories, in order
to maintain a realistic design with an alternation of text,
data visualization and other elements (e. g., illustration,
photograph, video) in a sequential structure. Data visual-
ization in particular is often positioned close to the key
arguments of the article, in order to provide readers with
empirical evidence [6]. Illustrations are usually included
to provide readers with visual context and clues about the
story. Combining text, illustrations and visualizations, we
generated three different versions of the same article. Each
version contained the same base text (with some minor
modifications in condition A which are explained below),
while the selection of visual representations varied: Ver-
sion A contained only illustrations, version B contained
only visualizations, and version AB combined both illus-
trations and visualizations.
The visualizations in the article present information that
is not found in the text, leaving the illustration-only
condition unbalanced in terms of content. We solved this
by developing the contents equivalently between condi-
tions. That is, we took the information contained in the
visualizations of conditions B and AB and integrated it
as text in the illustration-only condition (A). This ensured
that regardless of the version and the way the information
was presented, readers had access to the same content. As
an example, in condition B and AB we included a map of
Santiago showing the amount of liters consumed per per-
son per day. This same data was added in a text paragraph
in condition A (for the details of this translation, see the
supplementary material). We applied the same strategy
in all situations where we observed content imbalances
between conditions.
3.2 Hypotheses
To address our general research question—What are the
effects of data visualization compared to illustration in data
stories on readers’ attitudes?—we formulated six hypothe-
ses, based on previous studies about persuasion, emotions,
and memorability:
H1: Just after reading the article (t1-t0) the effect on
attitude is higher in conditions with visualizations (B,
AB) compared to the one with illustration only (A).
H2: Just after reading the article (t1-t0) the effect on
emotions is higher in conditions with visualizations (B,
AB) compared to the one with illustration only (A).
H3: Just after reading the article (t1-t0) the correct re-
sponse rate is higher in conditions with visualizations
(B, AB) compared to the one with illustration only (A).
H4: One week after reading the article (t2-t0) the effect on
attitude is higher in conditions with visualizations (B,
AB) compared to the one with illustration only( A).
H5: One week after reading the article (t2-t0) the effect
on emotions is higher in conditions with visualizations
(B, AB) compared to the one with illustration only (A).
H6: One week after reading the article (t2-t0) the correct
response rate is higher in conditions with visualizations
(B, AB) compared to the one with illustration only (A).
3.3 Experimental design
To evaluate different versions (with ethics board approval),
we used a between-subjects design.We compared three ver-
sions of the article with the visual representations described
above. We decided not to include a text-only version for
two key reasons. The first is that we sought to measure dif-
ferences in effect between visual representations (visualiza-
tions and illustrations) and not to compare measurements
against text alone, based on other similar studies that have
done so previously (e.g [60]). The second is that our study
seeks ecological validity, and in a real context, long-read
articles usually combine different kinds of visual support
[61].
Each participant was randomly assigned to one of the
three versions of the article. We divided the study into three
separate observational phases (see Figure 3). Participants
had to answer the first phase before (t0) and the second
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 6
one right after (t1) reading the article. After 5 days, we
sent them an email with a link to complete the study. They
returned to answer the third phase one week late on average
(t2).It is important to mention that this study is based
on a single article with three versions, so its results are
specific to the content presented. For external validity, it
will be necessary to repeat this study comparing articles
with different topics. However, we are moving toward a
better understanding of visual representations’ effects on
data stories.
3.3.1 Apparatus and participants
A call for participants was spread over social media. Au-
thors used their institutional and personal profiles across
various platforms to involve potential readers. Participants
had to be at least 18 years old and live in Chile. The entire
study was conducted in Spanish. In total, we recruited 198
participants, 133 of them fully answered t0 and t1, 71 also
answered t2. Among the total of 133 participants, 61 identi-
fied as women, 70 as men, and 2 as gender-diverse. In their
educational background, 58 participants reported a high
school degree, 32 participants reported a college degree, and
41 a graduate degree. The participants’ ages ranged between
18 and 67 years. They were distributed as follows: 66 people
aged 18-25 years, 27 between 26-36, 29 between 36-55, and 11
between 56-67 years. The participants’ occupations fell into
various categories: students (69), academics (8), architecture-
design (22), technology-engineering (13), social sciences (9),
and others (12). Participants used their own personal com-
puters to read the article and answer the questionnaire. The
article was blocked for mobile access. There was no financial
compensation for participating. All participants voluntarily
took part in the study.
3.3.2 Procedure
Participants accessed the page by clicking a link and accept-
ing an informed consent page, after which they began the
study.
• t0 — before reading the article: The initial questionnaire
included 10 questions in total. Four concerned demo-
graphics such as participants’ age, gender, education, and
occupation. An additional field asked to store partici-
pants’ email for the follow-up questionnaire at t2. One
question concerned the initial attitude of the participant
about the topic, followed by a rank of water crisis causes.
A third question assessed their emotional drive regarding
the water crisis topic. We ended by asking three questions
about their involvement concerning the topic. After this
first questionnaire, participants were randomly assigned
one version of the article and asked to carefully read it
through.
• t1 — just after reading the article: After reading the
article, participants had to answer the second question-
naire consisting of 9 questions. This evaluation round was
structured by combining old questions from t0 with new
ones. We started by re-assessing participants’ emotional
drive towards the topic. We then formulated four closed
questions with specific types of information they retained,
according to each section of the article. We drew from t0
again and asked participants about their attitude towards
the topic, followed by a ranking where they had to order
the causes of the water crisis. Finally, participants were
presented with two open questions. The first asked them
to reflect on the images presented in the article, the second
one prompted them to leave suggestions and comments.
After finishing t1, participants could leave the page. After
5 days they were notified about the third evaluation phase
(t2) via the email they provided during t0.
• t2 — about a week after reading the article: Participants
received a personalized link to complete the final ques-
tionnaire. The time lapse between participants answering
t1 and t2 varies, since they decided whether and when
to complete the study, and when they decided to return
to the third and last step. In this phase, participants
did not read the article again before completing the six
questions of the questionnaire. At this step, we repeated
the questions asked in t1, and included an open-ended
question about what they recalled from the article.
We collected time data for each phase of the study with sim-
ilar results across conditions; A (t1-t0: 27.96 minutes, t2-t1:
6.9 days), B (t1-t0: 25.35 min., t2-t1: 8.3 days), AB (t1-t0: 26.54
min., t2-t1: 8.1 days). During the study, participants were
limited and could not navigate back to previous pages. This
ensured that respondents’ answers were neither reviewed
nor changed from t0 and t1.
3.4 Measures
The questions were formulated to measure the attitudinal
effects of visual representations along three axes. For each
one, we applied methods extensively used in prior studies.
Change in attitude: In their research, Pandey et al.
[28] developed a method to effectively measure attitude
changes linked to data visualizations. The authors included
a question to measure attitudes on a given topic before
and after administering persuasive stimuli. Questions vary
according to the topic, but they are always formulated in
the same way: “to what extent do you agree that [...]” followed
by topic-specific statements. They measured answers on a
Likert scale ranging from -3 to +3. In our study, the question
to measure attitude was asked in all phases, posed as: Given
what you know about the water crisis in Chile, to what extent
do you agree (or disagree) that it can be solved? We used t0
as an initial benchmark to measure participants’ prior in-
volvement with the Chilean water crisis. Moreover, since we
created journalistic content in a long-read format, we added
a second question to find the reasons that may produce
changes in attitude. This question enlisted potential causes
for water scarcity, which readers had to rank according to
perceived relevance. Again, the question was repeated in all
phases. To avoid ordering biases, options were shuffled and
presented to readers in a random order.
Change in emotions: Psychology scholars have devised
a wide variety of instruments to measure emotions. One
of the most widespread is the Positive Affect and Nega-
tive Affect Schedule (PANAS) [62], which consists of a 20-
item self-reported questionnaire to measure both positive
and negative emotions. Among the various versions, we
chose the International PANAS short form (I-PANAS-SF)
[63] because of its brevity, with only 10 items. This tool
describes adjectives associated with habitual feelings and
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 7
emotions; upset, hostile, alert, ashamed, inspired, nervous,
determined, attentive, afraid, active. The response format
is a Likert scale, with a range from 1 (not at all) to 5
(extremely). For our study, carried out in Spanish, we used
the validated Chilean translation of PANAS [64]. In the
questionnaire, we applied the question as:“According to what
you know about the water situation in Chile, how do you feel about
this reality?” We asked the same question in the three phases
t0, t1 and t2. This allows us to compare the change in
emotions produced by different treatments between t0 and
t1, and subsequently between t0 and t2.
Comprehension and information retention: After read-
ing the article (t1), participants were asked about the con-
tent. We developed four questions to assess information
comprehension and retention. We included the same ques-
tions in t2, to compare the rate of correct answers.
3.5 Qualitative evaluation
We collected participants’ responses to the article content
and format through open-ended, free-form questions in t1
and t2. Immediately after reading the article, participants
were asked open-ended questions about the visual repre-
sentations’ utility (t1-OQ1), and whether they wanted to
share any other comments (t1-OQ2). We chose utility as
a framing to encourage participants to express their own
values in the context of data visualizations [46]. One week
later, participants were asked to recall any story elements—
image, idea or a particular bit of information (t2-OQ). Of the
133 valid participants in this study, 129 answered question
(t1-OQ1), while only 72 answered question (t1-OQ2). In
the following phase of the study (t2), 64 answered question
(t2-OQ).
We began to examine the responses by defining guiding
questions to help the research team decide what we wanted
to know: What were the effects of visual representations
in data stories on readers? Can we compare differences
between the two types of visual representations: illustra-
tions (A), visualizations (B), or the use of both (AB)?. In t1,
we sought to identify the visual representations’ immediate
impacts on participants’ reading experience. Analysis of the
responses in t2 would tell us about the lasting impact
on readers’ memory of the visual representations and the
content of the text. We grouped the answers according to
their study phase (t1 and t2) and the article version the
respective participant had read (A, B or AB). The analysis of
each answer set was carried out independently.
The research team generated a categorization scheme
for each of the answer sets, and used those schemes to
code the participants’ responses. This was complemented by
frequency analysis around the occurrence of each code [65].
We used the following process:
1) Two members of the team iteratively analyzed the
answers to code and categorize them until we believed
that a saturation point was reached.
2) Independently, a third coder used the categorization
proposed in the previous step to encode all the answers.
During this process, few new codes were identified.
3) This coding system was returned to the two team mem-
bers mentioned above, who reviewed it and generated
the final categorization of the answers.
4 RESULTS
Overall, we collected 133 answers for phase (t1-t0) and
71 answers in the follow-up survey at (t2). Subjects were
divided into three groups balanced by type of treatment;
group A (illustration only) N
A
=43, group B (visualization
only) N
B
=46 and group AB (illustration & visualization)
N
AB
=44. We conducted a post hoc power analysis using
GPower software, assuming an α-level of .05, an effect size
of .27 on attitude change and a statistical power of 80%.
The effect size of .27 was estimated based on a previous
study [28], which compared the attitude change of tables
versus charts on three topics and found effects between .214
and .538. Thus, we determined that a sample size of 75
subjects per cell was needed to detect a significant effect,
which made our study slightly underpowered (71 answers).
In the case of emotions, we did not conduct either an a priori
or a post-hoc power analysis, so we can not claim that our
study had enough power to detect significant effects related
to emotions. 5 days after participating in the study, we sent
the participants a link to fill the follow-up survey (t2). Par-
ticipants two days to answer it on average, i.e., about a week
after (t1). We also added three questions at the beginning
of the study (t0) regarding readers’ involvement in the
water crisis. This was a way to identify differences among
readers that could affect the results. The results showed
no differences between conditions. To ensure that readers
were paying attention, we observed the measurements we
conducted on information retention as well. As shown in
plots (j) and (k) in Figure 4, readers mostly gave 2 or 3
correct answers in each of the conditions. This suggests that
readers were paying attention when reading the articles.
The final number of participants who completed the full
study (t0, t1 and t2) was: N
A
=24 (44.1% attrition), N
B
=26
(43.4% attrition) and N
AB
=21 (52,2% attrition). This dropout
rate is reasonable for analysis, considering that the average
dropout rate reported in Web-based health interventions
was close to 50% [66]. The quantitative and qualitative
analyses conducted on the responses in two phases are
described below.
4.1 Quantitative analysis
We base our analysis on 95% bootstrap confidence intervals
(CIs) [67]. CIs are frequently used in visualization studies
[68], [69], [70], [71] and are recommended over p-values [72].
We analyzed the change in subjects’ attitude towards the
water crisis and their reported emotions before the study
and immediately after study (t1-t0), a week after the
study (t2-t0), as well as performance in questions mea-
suring information retention just after the study (t1) and
a week later (t2). Figure 4 shows the normalized sample
means together with 95% BCa CIs based on 10,000 bootstrap
replicates. For the interpretation of the statistical significance
of the overlap of CI error bars we refer to Krzywinski
and Altman [73]. All the detailed results can be found in
the supplementary materials together with the statistical
analysis.
4.1.1 Attitude
We compare the change in attitude through the question
Given what you know about the water crisis in Chile, to what
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 8
Attitude
Change
Legend
Information
Retention
Change in reported
emotions
(t2-t0): Change in the results
obtained between t2 (a week later of reading
the article) and t0 (before reading the article)
(t1-t0): Change in the results
obtained between t1 (just after reading the
article and t0 (before reading the article)
AB: Condition with illustration & visualization
B: Condition with visualization
A: Condition with illustration
Perception change in ranking
of causes of water crisis
A
B
AB
(t1-t0)
A
B
AB
(t2-t0)
d) Change in inefficient water use
−2 −1 0 1
A
B
AB
−2 −1 0 1
a) Change in attitude towards the water crisis
(t1-t0)
A
B
AB
(t2-t0)
A
B
AB
(t1-t0)
A
B
AB
(t2-t0)
−1.0 −0.5 0.0 0.5 1.0
e) Change in upset
A
B
AB
(t1-t0)
A
B
AB
(t2-t0)
f) Change in inspired
−1 0 1 2
A
B
AB
(t1-t0)
A
B
AB
(t2-t0)
g) Change in active
−1 0 1 2
A
B
AB
(t1-t0)
A
B
AB
(t2-t0)
h) Change in hostile
−1 0 1 2
A
B
AB
(t1-t0)
A
B
AB
(t2-t0)
i) Change in attentive
−1.0 −0.5 0.0 0.5 1.0
A
B
AB
j) Total number of correct answers in t1
0 1 2 3
A
B
AB
k) Total number of correct answers in t2
0 1 2 3
A
B
AB
(t1-t0)
A
B
AB
(t2-t0)
b) Change in water management and institutional deficiencies
−2 −1 0 1
A
B
AB
(t1-t0)
A
B
AB
(t2-t0)
c) Change in outdated water-related laws
−2 −1 0 1
Fig. 4. (a) Change in attitude changes towards the possibility of a solution to the water crisis for each condition. (b), (c) and (d) (e) Perception
change in ranking of water crisis causes. (f), (g), (h) and (i) Change in reported emotions of “upset”, “inspired”, “active”, “hostile” and “attentive”,
respectively. Plots from (a-i) reported each condition (A, B, AB) comparing between (t1-t0) as well as (t2-t0). (j) Total number of correct answers
per condition in t1 (just after reading the article). (k) Total number of correct answers per condition in t2, one week after the study took place. We
observe no significant difference among conditions, but a slight trend showing that condition AB (illustration & visualization) increases retention
compared to conditions A (illustration only) and B (visualization only).
extent do you agree (or disagree) that it can be solved ? We
asked this question in t0, and repeated in t1 and t2. This
allowed us to compare the change between them, thereby
identifying attitudinal changes regarding the subject. The
hypotheses on attitude change stated that the effect is higher
on conditions with visualization (B & AB) compared with
the condition with only illustration (A) in both hypotheses
(H1) immediately after reading the article (t1-t0) and
(H4) a week later (t2-t0). The results show that H1 and
H4 are supported only with condition B. In other words,
when comparing different conditions, we see that the article
with only visualizations (B) has a greater effect on readers
just after reading it (t1-t0), and it persists a week later
(t2-t0), producing a greater change of attitude regarding
the water crisis in Chile.
Figure 4 (a) reports the results of the primary question
measuring a change in attitude towards the possibility of a
solution to the water crisis. It represents the change among
the results obtained between t1 and t0 and then between
t2 and t0 for each condition. When we analyse condition
A, we do not see changes between t1 and t0 or between t2
and t0, since both the CIs include 0 as shown in Figure
4(a). In other words, there was no significant attitudinal
change in the condition with illustrations only (A), neither
immediately after reading the article (Mean (A(t1-t0))= 0.045,
95% CI = [-0.23, 0.23]), nor a week later (Mean (A(t2-t0))=
0.273, 95% CI = [0.00, 0.45]). Meanwhile, in condition B
we observed a significant change immediately after the
treatment (Mean (B(t1-t0))= -0.440, 95% CI = [-1.16, -0.12]).
That is, after reading the article with visualizations we ob-
served an effect of condition B as a change in the perception
regarding the solution of the water crisis. This change in
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 9
attitude has a negative trend, meaning that after reading the
article, participants believe that there is smaller chance of a
solution to the water crisis. After one week, we observe that
this effect holds on subjects under condition B (Mean (B(t2-
t0))= -0.600, 95% CI = [-1.52, -0.12]). Finally, in condition AB
there is no significant change in attitude right after reading
the article with both illustration and visualization (Mean
(AB(t1-t0))= -0.158, 95% CI = [-1.37, 0.32]). Likewise, we did
not observe changes after a week (Mean (AB(t2-t0))= -0.526,
95% CI = [-1.63, 0.00]), but we noticed a larger variance in
attitude change compared to the small variance in condition
A, already mentioned above.
To inquire about the reasons that may produce the
change of attitude, in a second question we asked readers to
rank five causes of the water crisis: “drought and decreased
precipitation”, “water management and institutional defi-
ciencies, “outdated water-related laws”, “inefficient water
use”, and “inequality in access to water”. The results show
significant change in three of the causes as shown in Figure
4 (b-d). Readers deem the cause “water management and
institutional deficiencies” (Figure 4 (b)) to be more relevant
to the water crisis after reading the article. It increases
in importance (ranking) significantly just after reading the
articles (t1-t0) containing visualizations; For condition B
(Mean (B(t1-t0))= -0.462, 95% CI = [-0.92, -0.038]) and for con-
dition AB (Mean (AB(t1-t0))= -1.150, 95% CI = [-1.65, -0.750]).
However, this change of perception does not hold after
one week (t2-t0) in any condition. Meanwhile, when we
observe the cause of “outdated water-related laws” (Figure
4 (c)) we notice that it increases in importance significantly
only in condition A (with illustration only) just after reading
the article (Mean (A(t1-t0))= -0.54, 95% CI = [-1.21, -0.083]),
and this effect is maintained a week later (Mean (A(t2-t0))=
-0.75, 95% CI = [-1.54, -0.125]). Interestingly, the cause of
“inefficient water use” (Figure 4 (d)) decreases in impor-
tance just after reading the article only in condition A (Mean
(A(t1-t0))= 0.042, 95% CI = [-0.33, 0.292]) , but it does not
hold after one week (Mean (A(t2-t0))= 0.417, 95% CI = [-0.21,
0.875]).
Observing these results, we conclude that only condition
B produces a significant and persistent change in attitude,
but this effect cannot be explained only by changes in
perceptions of the reason behind the water crisis. Although
we observed that the cause ”water management and in-
stitutional deficiencies” produces significant differences for
condition B, it does not persist one week later (t2-t0).
Therefore, the next step is to analyze other dimensions that
can help us explain the change in attitude; emotions and
information retention.
4.1.2 Emotions
For this study, we measured 10 different emotions following
the instrument (I-PANAS-SF) (see section 3.4). The hypothe-
ses on the change in emotions stated that the effect is higher
on conditions with visualization (B & AB) compared with the
condition with only illustration (A) in both (H2) immediately
after reading the article (t1-t0) and (H5) a week later
(t2-t0). The results support H2 only for the emotion
“active”. However, when we observe condition B separately,
(H1) we see it is also supported for the emotions “hostile”
and “attentive”. H4 is also supported in condition B, for
Fig. 5. Summarizes the effects on change in attitude, emotion, and
information retention observed when comparing the three conditions
(A, B & AB) in the two phases of the study (t1-t0) and (t2-t0).
Primary (P) and secondary (S) questions were used to measure attitude.
the emotions “inspired”, “active”, “hostile” and “attentive”.
When comparing different conditions, we thus see that the
articles with only visualizations (B) have a higher effect
on readers just after reading the article (t1-t0) and that
this effect also persists after a week (t2-t0), producing
a greater change of emotions compared to conditions with
illustrations (A and AB).
We observed important changes in 5 of the emotions we
measured, which appear in Figure 4 (e-i). Figure 4 (e) shows
the changes produced in “upset”. The effect corresponds
to the change reported by readers just after reading the
article (t1-t0) and by a week later (t2-t0). The “upset”
emotion increased just after reading the article (t1-t0) in
the three conditions, and vanished over time in all three
conditions. The mean changes of the emotion ”upset” for
the three conditions are the following: For condition A in
(t1-t0) (Mean (A(t1-t0))= 0.455, 95% CI = [-0.091, 0.68])
and in (t2-t0) (Mean (A(t2-t0))= 0.091, 95% CI = [-0.227,
0.32]); for condition B in (t1-t0) (Mean (B(t1-t0))= 0.500,
95% CI = [0.192, 0.88]) and in (t2-t0)) (Mean (B(t2-t0))=
0.192, 95% CI = [-0.231, 0.73]); finally in condition AB in
(t1-t0) (Mean (AB(t1-t0))= 0.238, 95% CI = [0.048, 0.38])
and in (t2-t0) (Mean (AB(t2-t0))= -0.286, 95% CI = [-0.714,
0.00]).
On the other hand, Figure 4 (f) illustrates the changes
produced in “inspired”. Here we see that after reading the
article (t1-t0) participants felt more inspired in all con-
ditions. However, unlike “upset”, the effect on this emotion
remained stable after a week (t2-t0) only in conditions
with visualization (B & AB) since we observe that the CI
bars for these conditions do not touch 0. The mean changes
of the emotion ”inspired” for the three conditions is the
following: For condition A in (t1-t0) (Mean (A(t1-t0)=
0.45, 95% CI = [0.136, 0.82]) and in (t2-t0) (Mean (A(t2-
t0))= 0.41, 95% CI = [-0.091, 0.86]); Meanwhile for condition
B in (t1-t0) (Mean (B(t1-t0))= 0.81, 95% CI = [-0.423, 1.15])
and in (t2-t0) (Mean (B(t2-t0))= 0.58, 95% CI = [0.269,
0.92]); Finally for condition AB in (t1-t0) (Mean (AB(t1-
t0))= 0.74, 95% CI = [0.421, 1.00]) and in (t2-t0) (Mean
(AB(t2-t0))= 0.79, 95% CI = [0.368, 1.26]).
When analyzing the emotion “active”, we see that there
is a change between the conditions with visualization (B
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 10
& AB) and the one with only illustrations (A). Figure 4 (g)
shows that this emotion increased in readers, after reading
the article, in condition B (Mean (B(t1-t0))= 0.87, 95% CI =
[0.58, 1.09]) and in condition AB (Mean (AB(t1-t0))= 0.45, 95%
CI = [0.05, 0.75]). After one week this state was maintained
only for condition B (Mean (B(t2-t0))= 0.74, 95% CI = [0.39,
1.00]). The emotion “attentive” as shown in Figure 4 (i),
increased just after reading the article only in condition B
(Mean (B(t1-t0))= 0.417, 95% CI = [0.167, 0.67]) and remained
a week later, (Mean (B(t2-t0))= 0.417, 95% CI = [0.042, 0.75]).
When analyzing the remaining options (alert, ashamed,
nervous and determined) we did not find significant
changes. The result of this analysis can be found in the
supplementary materials (section 2.1.2). As a summary,
Figure 5 shows the conditions in which we see changes for
each measurement in the study. Looking at the changes in
each of the emotions reported between the different phases
of the study, we see that condition B (with visualization
only) shows changes for five emotions just after reading
the article (t1-t0). In other words, the condition with
only visualizations triggered a higher emotional response
compared to the conditions with illustration (A & AB). On
the other hand, the effect on these emotions was persistent
one week later (t2-t0) for all emotions in which a change
is observed, except for the emotion ”upset”.
4.1.3 Information retention
The retention results show no significant effect from the
different types of conditions with which participants in-
teracted. Just after finishing the study t1, as shown in
Figure 4 (j), we see no significant changes in the number
of correct answers: For condition A (Mean (A(t1))= 2.2, 95%
CI = [1.9, 2.5]); for condition B (Mean (B(t1))= 2.4, 95% CI =
[1.9, 2.7]) and finally for condition AB (Mean (AB(t1))= 2.7,
95% CI = [2.3, 2.9]). When analyzing the results of these
same questions on t2 we observed a slight decrease in
the number of answers, but it is uniform across conditions
and non-significant. Moreover, in t2 no condition showed
significantly better information retention than the others,
either observing each question in isolation or in aggregation
as shown in Figure 4 (k): For condition A (Mean (A(t2))= 2.3,
95% CI = [1.9, 2.5]); for condition B (Mean (B(t2))= 2.4, 95%
CI = [2.0, 2.6]) and finally for condition AB (Mean (AB(t2))=
2.5, 95% CI = [2.2, 2.8]) In summary, the results do not
appear to support H3 and H6.
4.2 Qualitative analysis
This section presents major themes observed in the
study questionnaires’ free-form responses. After analysis of
the question regarding visual representations’ usefulness
(t1-OQ1), 13 categories emerged, as shown in figure 6.
These allow us to characterize visual representations’ role
for readers in better understanding or contextualizing the
textual content. We then organized the responses into the
elements remembered and the types of comments readers
made about the data story in general. After each sub-
category, we added a number in parenthesis representing
their occurrence in the responses. We included some illustra-
tive quotes from participants in the most relevant findings
of this study as well. Quotes are reported and translated into
English, with participant number and condition, e.g., [P123,
AB]. OQ1 stands for Open Question 1.
4.2.1 Roles of visual representations
When analyzing responses to the question about the utility
of the visual representations (t1-OQ1) the first thing we ob-
serve is that most readers in condition B explicitly stated that
the visual artifacts were useful and impactful (39 out of 44).
In condition AB answers are similar to condition B: 33 out of
42 readers found visual artifacts useful for their reading pro-
cess. This differs from condition A where more than half of
the readers (22 out of 40) deemed illustrations not or hardly
useful for understanding the content. When analyzing these
responses, different categories emerged which allowed us
to identify the functionality that visual representations have
for readers. We identified 9 categories that we consider
positive and 4 negative. Across the three conditions, visual
artifacts are described as having positive effects; they often
have an aesthetic appeal, they draw readers’ attention, and
they provoke emotional reactions. However, they are also
associated with negative descriptions at times, considered
confusing or distracting, they can cause readers to feel frus-
tration, or they are just forgotten. For instance, in condition
A visual artifacts are found evocative (13) and supporting
(16):
“The illustrations are beautiful, [...] I think they are
useful [...] They complement each other very well and
make the text easier to read” [P14, A]
Readers consider them as a good addition to the text,
helping them with placing and visually representing the
concepts discussed in the text. However, artifacts in con-
ditions A are not essential for readers, who do not rely on
them to fully understand written content:
“[...] I don’t think [the illustrations] were particularly
useful for understanding the text” [P113, A]
On the other hand, while readers consider illustrations
to be beautiful (4), they often forget (7) about them almost
immediately, or even find them confusing (4) and not well
connected to text:
“[Illustrations] were very unimportant, as they didn’t
present a clear idea and didn’t complement what the
text said. [...] they do not help to better explain the
information.” [P84, A]
When we observe the responses in condition B, readers
consider the role of the visual artifacts as educational (12),
clarifying (10), intelligible (7) and evocative (8). Charts are
discussed as highly understandable and responsible for
carrying essential information beyond the text. Their clarity
helps readers in forming context and in changing attitudes
or enriching their knowledge regarding the water crisis:
“I think [the visualizations] were useful because they
explained, highlighted and allowed the reader to compare
the data shown in the text, which sometimes don’t seem
important or are easy to forget.” [P82, B]
The statements about the visual representations in condition
B are also largely favorable. Apart from receiving positive
comments on their aesthetic presentation (6), visual artifacts
are considered to have a remarkable influence on readers’
attention (8) as well as on their emotional response (8).
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 11
While negative effects such as confusion (3) and frustration
(2) are present, their impact seems to be minimal compared
with the positive aspects:
“They were useful for understanding the information,
because visualizing water shortage is much more shock-
ing than reading about it in the text.” [P8, B]
Similarly, in condition AB readers describe artifacts as educa-
tional (12), clarifying (10), intelligible (8) and evocative (7).
Notably, for this condition, scores are almost identical to
condition B. This is often due to how visualizations receive
more attention and feedback. Readers exclusively refer to
visualizations on 20 occasions. By contrast, illustrations are
never discussed alone, but only in combination with or in
subordination to visualizations:
“[Visualizations] give a clearer image of all the affected
sectors, and the implications of the reduction in water
reserves [...]” [P66, AB]
Finally, in condition AB we face mixed reports from readers.
It is difficult to define which effects are relevant, since
the responses are quite diverse. Some of them considered
artifacts to be aesthetically pleasing (4). When given vi-
sual representations of data, readers also reported feeling
more impacted and thrilled (5) about the subject. However,
negative effects seem to be slightly more incisive. Artifacts’
density and interactivity is also deemed to be frustrating (5),
while some visual elements are considered to be confusing
(5).
“Some of them were quite useful and helped you not to
get so tired reading; [...] to start with it was interesting,
but after a while it made you a bit dizzy [...]” [P80, AB]
Fig. 6. Summary of the 13 categories that emerged from analyzing the
open-ended responses (t1-OQ1) about visual representations’ useful-
ness. These categories are organized by positive (+) and negative (-),
where: (Be: Beautiful), (At: Attention-grabbing (Ev: Evocative), (In: Intel-
ligible), (Me: Memory), (Ed: Educational), (Cl: Clarifying), (Th: Thrilling),
(Su: Supportive), (Co: Confusing), (Di: Distracting), (Fr: Frustrating) and
(Fo: Forgettable). Circle size represents these categories’ occurrences
in each of the conditions (AB, B, A).
4.2.2 Comments
With open question (t1-OQ2) the first thing we notice
is that comments can be categorized into those referring
to the contents of the article and those that refer to the
format (visualizations, illustrations, interaction, etc.). In both
categories we find opinions that can be categorized as sug-
gestions, criticisms, and compliments. In all three conditions
the comments mainly correspond to the format of the article.
We see that condition A receives more comments on the
contents than the others (A=11; B=5; AB=3). Most of their
comments are content focusing on possible solutions and
problems related to the water crisis, pivoting around con-
crete solutions and the will of the reader to discuss positive
actions, and not only negative consequences. We found that
these types of comments about the content correspond more
frequently to suggestions and compliments, rather than
criticism.
“[...] While [the article] doesn’t seem to talk about
management strategies to take care of water, I would love
to see it address the impact that better city design, urban
and rural irrigation systems could have [...]” [P14, A]
On the other hand, the open-ended comments in conditions
AB and B, refer more often to the format of the article. Al-
though these comments correspond to suggestions, compli-
ments and criticisms, we can observe that there is a tendency
towards the latter. Condition B gathers the largest number
of negative comments, usually targeting visual aspects of
the charts or interaction features.
4.2.3 Recalled elements
When analyzing the answers to question (t2-OQ), which
was about recalling any story elements, we identified five
sub-categories; illustrations, visualizations, textual content,
emotions and specific data. Conditions A and AB present
four illustrations associated with each of its sections. One
example of the description is as follows:
“[...] Picture of mountains and drought in a mine –
comparison of a garden with a swimming pool with one
without water – President’s desk with the ‘leaky system’
[...]” [P7, A]
In condition AB, only one response mentions an illustra-
tion. When comparing both conditions (A and AB) we can
observe that illustrations were less remembered by readers
when they were presented together with visualizations. In
condition B and AB, visualizations of different characteristics
were presented: maps, graphs and diagrams. We do not ob-
serve many differences between what the readers remember.
They all mainly described maps (16) and to a lesser extent,
graphs (7) and diagrams (5). For example:
“the map of Santiago with the historical annual rain-
fall [...]” [P21, AB] .
Regarding the textual content of the article, in condition A,
we observe that readers remembered it more frequently (14)
while in condition B (7) and AB (10) it was somewhat lower.
The destruction of glaciers is the topic with the highest recall
across the three versions.
Finally, we also observed some responses about emo-
tions that participants remember. Although there are not
many (A=1; B=3 and AB=3) the statements in conditions B
and AB are: “shocking”, “devastating” and “frustration”:
“I remember my feeling of powerlessness as I read the
article. It was worse when the article visualized the
number of people in Chile who are supplied by water
trucks. [...] and the fact that a large number of the
population do not have access to water is shocking
[...]” [P13, AB]
Akin to the responses to question (t1-OQ), this suggests that
the article versions with visualization elicit more emotional
responses compared to the condition with illustrations only.
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 12
5 DISCUSSION
Our results show that data-driven long reads, where infor-
mation is visually rendered only by charts, have a higher
impact on readers’ attitude compared to long reads fea-
turing illustrations. These findings pave the way towards
further research on the effect of visualizations compared to
illustrations in data-driven storytelling. We observe that the
condition with only visualization (B) produces a change in
attitude towards the water crisis just after reading the article
which persists one week later, although with a decreasing
effect size, as opposed to conditions with illustrations (A &
AB). Similarly, our results show that the visualization-only
condition (B) triggered the greatest number of emotions.
With information retention, we did not observe significant
differences among conditions.
The answers to free-form questions also offer insights on
how readers approached artifacts across the three versions
of the story. In regards to A, illustrations were neither
considered particularly useful nor effective to understand
the content. Readers found them beautiful, but struggled to
retain them in the long term. This condition also elicits more
content-oriented comments. Version B instead scored well
when it comes to utility: readers considered visualizations
an essential component of the story. They consequently also
seemed to be more prone to report positive influences on
their attention span and sense-making process. As opposed
to illustrations, readers often remembered visualizations
over text and were more likely to comment on them in
the later phases of the evaluation. With version AB, it is
interesting to notice that, although visualizations and il-
lustrations were mixed, readers’ answers were very similar
to the ones for version B, suggesting the prominent role of
visualizations over illustrations. Overall, illustrations seem
to be less memorable and functional than visualizations.
5.1 Implications
When considering the results obtained in both the statistical
and qualitative analysis, we reflect on the persuasive effects
observed in the condition that include only visualizations
(B). While this requires further analysis and experiments, we
speculate that this effect is related to the emotional impact
triggered by the visualizations, as well as being explained
by the educational, clarifying, and intelligible role readers
attributed to them. Through these roles, visualization be-
comes a tool facilitating readers’ access to the information
by making it more comprehensible. This suggests that data
visualization is a more persuasive support for an argument
than illustration alone. After observing the effects reported
by readers under condition B, we found it revealing that
visualization elicited more emotional responses. We suspect
that visualization clarifies the information presented, gener-
ating an emotional connection and persuading the reader.
This concurs with the idea that attitude change can be
achieved through analytical reasoning, due to the evidence
in the qualitative analysis, or emotional appeal based on the
evidence in the quantitative statistical analysis [38].
Analyzing the impact produced by different types of
visual representations showed us some patterns which,
while they require further investigation, may have profound
implications for their use in data stories. First, when we
look at the changes in the order that readers gave to the
causes of the water crisis, we see that the cause of “water
management and institutional deficiencies”, rose right after
reading the article in the conditions with visualization (B &
AB). This can be explained by the use of a particular diagram
used in conditions B and AB (see supplementary material),
which represents the number and variety of actors involved
in water management. This diagram may emphasize that
particular idea. In the illustration-only condition there is no
visual representation referring to this cause, and instead it
is only presented as text. On the other hand, when looking
at the cause of ”outdated water-related laws” we see that
it rises in the ranking in the condition with illustration
only. By contrast with the cause ”water management and
institutional deficiencies”, this cause has no visual repre-
sentation (in any of the conditions) and is only referred
to in the text. While this requires further investigation, we
can say that readers in the A condition paid more attention
to the text, and that on the contrary, readers in the B and
AB conditions were persuaded by the visual representation
associated with the “water management and institutional
deficiencies” cause. Along the same line but focusing on
data retention, when comparing the three study conditions
we did not observe differences in the effects. However, in the
answers to the open questions (t1-OQ2) and (t2-OQ), the
readers in the condition with illustrations only (A) alluded
mainly to the content of the article; while those in conditions
with visualization (B & AB), mentioned the visualizations
much more frequently. These two different findings could
be important for designing a data story: Do we want readers
to concentrate on the content of the article, or the form? If
we want readers to focus on the article for its content rather
than its visualizations, these may need to be less prominent.
Complex arguments can be enhanced by visualizations,
and by strategically choosing where to place and how to
combine them. Reader retention may be stronger or weaker
as a result. In any scenario, this aspect requires further
investigation.
We find it relevant to the implications for visual rep-
resentations use to observe the differences in the emotion
”active” as well (see Figure 4 (d)). Here, we see that the
effect on emotion change is larger in the conditions with
visualization (B & AB) after reading the article, and persists
one week later in the condition with visualization only. This
result suggests that visualizations act as a “call to action”,
which might be relevant for authors of data stories who seek
to trigger readers’ actions toward the subject presented.
5.2 Limitations
Although our approach is effective and our results are
promising, some limitations should be considered.
5.2.1 Measuring emotions
According to our qualitative analysis, visualizations in par-
ticular elicit different emotional responses among readers
compared to the emotion measured in the questionnaire of
the study. This may be a result of the type of measurement
we used (see section 3.4). While the issues presented in a
data story vary widely, the instrument measures a limited
number of emotions. It is possible that it does not accu-
rately pick up the full spectrum of emotional responses
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 13
that the story may elicit. For example, in the qualitative
analysis some emotions reported by readers express: “frus-
tration”, “impact”, “shocking”, “devastating”. None of these
appear in the I-PANAS-SF questionnaire, though. Future
work could complement this measurement with others that
address specific emotions expected in response to a data
story. On the other hand, the I-PANAS-SF only focuses
on emotional valences, i.e. on positive and negative af-
fect. Future studies could incorporate a two-dimensional
space model spanning valence and arousal (intensity of
the emotion) [74]. It is important to keep the length of the
questionnaire limited, though, so as to avoid study fatigue
in the participants.
5.2.2 Measuring information retention
Taking study fatigue into account, we attempted to mea-
sure information retention in a reduced setting. To make
it less tedious for the participants, we used as few ques-
tions as possible. However, in light of the results obtained
in the statistical analysis, we suspect that the questions
that we formulated were limited to very specific contents,
which were not necessarily sufficient to identify what the
readers remembered more generally. The open questions
included in both t1 and t2, on the other hand, provided
us with much more detailed information on what readers
remembered. This suggests that future work should adapt
and/or synthesize a procedure such as the one proposed by
Bateman et al. [8] in their study on visual embellishment
and memorability. In their work, a more detailed picture
was gained from interviews about what the participants
understood regarding the visualizations that they were
shown, and what they remembered immediately afterwards
or some weeks later. Although an interview is longer for the
participants, and takes the researchers longer to analyze, it
yields more detailed quality information. Depending on the
length of the study, it needs to be balanced to obtain more
complete measurement without generating fatigue among
the participants.
5.2.3 Visual representations ratio, dynamism, and novelty
There are some considerable differences between how vi-
sualizations and illustrations are presented to our study
participants: ratio and dynamism. The ratio between charts
and illustration is not one to one, meaning that we included
a higher number of charts (12) than illustrations (4). This
asymmetry could potentially bias our participants to eval-
uate visualizations as more informative simply based on
quantity. However, our results show that participants in B
almost never consider smaller charts, and focus exclusively
on main visualizations at the beginning of each section.
Nevertheless, future studies should better consider the ratio
of visual representations and ensure full symmetry among
different settings. For dynamism, some visualizations ben-
efit from animations, while the illustrations were entirely
static. This is due to how different formats usually present
information. ”Scrollytelling” has become a common and
recognized technique in data stories, and it is fairly common
for visualizations to be dynamic. On the other hand, illus-
trations are often single static objects. Future work should
concentrate on this difference, since animations could be
more attractive for readers and ensure a similar treatment
for different types of visual representations.
While visualizations are becoming more common in the
context of journalism, they still require a different level
of data literacy in comparison to static illustrations. The
relative novelty of visualization compared to an assumed
familiarity with illustrations may have some impact on
persuasion and information retention.
Another aspect that should be observed is the particular
characteristics and qualities of the illustrations and visual-
izations, which could have a differential impact on reader
impressions of the topic. Poor-quality visual representations
could lead to readers misunderstanding the information,
thereby influencing the results of a study such as this. An
exceptionally high quality design may also create a positive
bias. Of course, each of the three conditions presents only
one article design, which could have looked very different if
created by another team in a different context. The purpose
of this study was to examine three real-world news articles,
each of which works as an informative data story about
environmental issues. We have made every effort possible
to create realistic and representative conditions; however,
we fully acknowledge that this does not and can not present
generalizable results covering other topics.
Another aspect to consider is the topic to be covered in
the article. Some topics may benefit more from one type
of visual representation than another, such as those that
require both data representation and process explanation
(e.g., creating a new vaccine, the effects of a hurricane,
etc.). These articles could benefit from a clear illustration
that explains a succession of facts. This opens up future
work that could be oriented towards identifying the most
appropriate visual representations (or combination of these)
for particular topics.
5.2.4 Low return rate
The return rate of participants between t1 and t2 was
relatively low. Since the third questionnaire is administered
a week later t1, participants often drop out. Such dropouts
could potentially jeopardize the last part of the study and
weaken our findings on memorability. Given the consistent
number of overall participants, the number of answers to
t2 is still enough to obtain preliminary results and carry
out qualitative analysis on free-form answers. Nevertheless,
future studies should consider possible ways to minimize
attrition, possibly by establishing a reward system for par-
ticipants or by effectively motivating them to come back and
finish the study.
5.2.5 Free-form response
The open-ended questions we asked in the study may
lead to a positive bias due to how we phrased them. For
instance, the question (t1-OQ1) was phrased as: Were (visual
representation) useful for understanding the information given in
the article? Although the wording was identical for all three
conditions, and its positive phrasing could uniformly extend
to all, we recognize that a more neutral phrasing might
reduce this potential bias. Further studies should consider
this limitation.
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 14
5.3 Future work
The results presented and discussed in this paper are a
promising starting point for similar studies investigating
other dimensions or comparing other types of visual rep-
resentations. One aspect that could be further investigated
is whether data visualizations activate readers to perform
certain actions. Given an explicit call to action (e.g. donation,
sign a petition, etc.), would visualizations make data stories more
actionable than illustrations? Investigating this aspect could
also help clarify why negative arguments generate positive
emotional activation as in the case of our quantitative and
qualitative results.
Another way forward could be to look at differences
between photographs and visualizations with a similar ap-
proach. For instance, we could ask: Do visualizations influence
readers’ attitude as strongly as photographs? If yes, how do they
concur to change beliefs and habits? This research branch could
even expand to false memories: How can visualizations manip-
ulate readers’ memories? This particular question is becoming
vital, in the light of repeated misuse of data visualizations
in agenda setting and fake news.
Although the results we obtained are promising, they
are based solely on one data story published in three
variants. We consider that adding entirely different data
stories and running a similar experimental study to test
similar hypotheses would add more external validity to
the conclusions. Future work could consider running a
similar experiment with multiple stories that have multiple
variations, for example, in textual length or topical scope.
One could also consider extending the measurement and
compare the differences between a text-only article and
versions with different visual representations (e.g. videos,
photos).
In this study, we found evidence of the effect of visual-
izations on attitude, but did not study in detail how the
positioning of text and visual artifacts (visualizations or
illustrations) may influence this outcome. The positioning
and distribution of visual representations and text within a
data story could influence reader attitude changes. Future
research could consider studying the impact that visuals
and their position may have, incorporating variables such as
the semantic and spatial proximity of visual representations
to text. Finally, in order to separate the effects of visual
conditions compared to previous knowledge of the topic,
we also suggest including questions to assess participants’
expertise about the water crisis.
6 CONCLUSION
As journalistic forms of storytelling change to accommodate
a wide range of contents including data, visualizations play
an ever-increasing role for informing about issues of public
concern. There has already been considerable research on
other visual representations in news media, but there has
been little evidence on visualizations’ effect on attitude
changes. With this research we attempted to get a better
understanding of the persuasive, emotive, and memorable
qualities of visualizations compared to illustrations when
embedded in long read articles. We researched the role of
visual representations via one particular data story about a
current topic that proved to be a promising strategy. We
formulated 6 hypotheses in order to test different visual
artifacts’ effects (visualization compared to illustrations and
their interaction) upon attitudinal change, emotions, and
information retention by conducting a user study. Our
main conclusions are summarized as follows. With this
research we contribute insights on the attitudinal effects
of visualization both in comparison to and in combination
with illustration. First, our results suggest that using only
visualizations within a data story can have a significant
effect on attitudinal changes. Second, they can also trigger
a higher number of emotions, which leads us to think that
the change in attitude may be related to the emotional im-
pact they generate, even though claiming this relationship
needs further research. Third, visual representation type has
no statistically significant effect on information retention;
however, the qualitative analysis has provided evidence
of a considerably higher level of detail and elaboration in
the free-form responses regarding articles which feature
visualizations. These are important indications about the
attitudinal effect of visualizations, in particular regarding
complex issues such as environmental crises. During this
research we came across multiple open issues, which we
address in our limitations and consider to be an interesting
starting point for future work. Thus, while our results are
not fully conclusive, we see a great need for this line of
research on the rhetorical power of visualizations in the
context of storytelling.
ACKNOWLEDGMENTS
This work was funded by ANID - Doctorate Grant and
partially funded by ANID, Millennium Science Initiative
Programs Code ICN17 002 (IMFD), ICN2021 004 (iHealth)
and by Basal Funds FB210017 (CENIA). The authors would
also like to thank Lena Zagora, M.Jesus Guarda, Arran
Ridley and Jonas Arndt.
REFERENCES
[1] E. Mitchelstein and P. J. Boczkowski, “Between tradition and
change: A review of recent research on online news production,”
Journalism, vol. 10, no. 5, pp. 562–586, 2009.
[2] F. Hohman, M. Conlen, J. Heer, and D. H. P. Chau, “Communicat-
ing with interactive articles,” Distill, 2020.
[3] R. Kosara and J. Mackinlay, “Storytelling: The next step for visu-
alization,” Computer, vol. 46, no. 5, pp. 44–50, 2013.
[4] N. H. Riche, C. Hurter, N. Diakopoulos, and S. Carpendale, Data-
driven storytelling. CRC Press, 2018.
[5] E. Segel and J. Heer, “Narrative visualization: Telling stories with
data,” IEEE Transactions on Visualization and Computer Graphics,
vol. 16, no. 6, pp. 1139–1148, 2010.
[6] W. Weber, M. Engebretsen, and H. Kennedy, “Data stories: rethink-
ing journalistic storytelling in the context of data journalism,”
Studies in Communication Sciences, vol. 2018, no. 1, pp. 191–206,
2018.
[7] L. Bounegru and J. Gray, The Data Journalism Handbook: Towards a
Critical Data Practice. Amsterdam University Press, 2021.
[8] S. Bateman, R. L. Mandryk, C. Gutwin, A. Genest, D. McDine,
and C. Brooks, “Useful junk? the effects of visual embellishment
on comprehension and memorability of charts,” in Proceedings
of the SIGCHI Conference on Human Factors in Computing
Systems, ser. CHI ’10. New York, NY, USA: Association for
Computing Machinery, 2010, p. 2573–2582. [Online]. Available:
https://doi.org/10.1145/1753326.1753716
[9] R. Borgo, A. Abdul-Rahman, F. Mohamed, P. W. Grant, I. Reppa,
L. Floridi, and M. Chen, “An empirical study on using visual
embellishments in visualization,” IEEE Transactions on Visualization
and Computer Graphics, vol. 18, no. 12, pp. 2759–2768, 2012.
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 15
[10] L. Byrne, D. Angus, and J. Wiles, “Acquired codes of meaning
in data visualization and infographics: Beyond perceptual prim-
itives,” IEEE Transactions on Visualization and Computer Graphics,
vol. 22, no. 1, pp. 509–518, Jan. 2016.
[11] J. Boy, A. V. Pandey, J. Emerson, M. Satterthwaite, O. Nov, and
E. Bertini, “Showing people behind data: Does anthropomorphiz-
ing visualizations elicit more empathy for human rights data?”
in Proceedings of the 2017 CHI Conference on Human Factors in
Computing Systems, 2017, pp. 5462–5474.
[12] B. Bach, N. Kerracher, K. W. Hall, S. Carpendale, J. Kennedy,
and N. Henry Riche, “Telling stories about dynamic networks
with graph comics,” in Proceedings of the 2016 CHI Conference on
Human Factors in Computing Systems, ser. CHI ’16. New York, NY,
USA: Association for Computing Machinery, 2016, p. 3670–3682.
[Online]. Available: https://doi.org/10.1145/2858036.2858387
[13] L. Rodriguez and D. V. Dimitrova, “The levels of visual framing,”
Journal of Visual Literacy, vol. 30, no. 1, pp. 48–65, 2011. [Online].
Available: https://doi.org/10.1080/23796529.2011.11674684
[14] H.-J. Bucher and P. Schumacher, “The relevance of attention for
selecting news content. an eye-tracking study on attention patterns
in the reception of print and online media,” Communications,
vol. 31, no. 3, pp. 347–368, 2006.
[15] L. J. Kensicki, “Deaf president now! positive media framing of
a social movement within a hegemonic political environment,”
Journal of Communication Inquiry, vol. 25, no. 2, pp. 147–166, 2001.
[16] C. King and P. M. Lester, “Photographic coverage during the
persian gulf and iraqi wars in three us newspapers,” Journalism
& Mass Communication Quarterly, vol. 82, no. 3, pp. 623–637, 2005.
[17] S. Parrott, J. Hoewe, M. Fan, and K. Huffman, “Portrayals of
immigrants and refugees in us news media: Visual framing
and its effect on emotions and attitudes,” Journal of Broadcasting
& Electronic Media, vol. 63, no. 4, pp. 677–697, 2019. [Online].
Available: https://doi.org/10.1080/08838151.2019.1681860
[18] T. E. Powell, H. G. Boomgaarden, K. De Swert, and C. H. de Vreese,
“A clearer picture: The contribution of visuals and text to framing
effects,” Journal of Communication, vol. 65, no. 6, pp. 997–1017,
2015. [Online]. Available: https://doi.org/10.1111/jcom.12184
[19] C. Brantner, K. Lobinger, and I. Wetzstein, “Effects of visual
framing on emotional responses and evaluations of news stories
about the gaza conflict 2009,” Journalism & Mass Communication
Quarterly, vol. 88, no. 3, pp. 523–540, 2011.
[20] J. Hullman and N. Diakopoulos, “Visualization rhetoric: Framing
effects in narrative visualization,” IEEE Transactions on Visualiza-
tion and Computer Graphics, vol. 17, no. 12, pp. 2231–2240, Dec.
2011.
[21] R. E. Petty and P. Bri
˜
nol, “Attitude change,” in Advanced social
psychology: The state of the science, R. F. Baumeister and E. J. Finkel,
Eds. Oxford University Press, 2010, pp. 217–259.
[22] R. E. Petty and J. T. Cacioppo, “The elaboration likelihood model
of persuasion,” in Communication and persuasion. Springer, 1986,
pp. 1–24.
[23] R. E. Petty and P. Bri
˜
nol, “Emotion and persuasion: Cognitive
and meta-cognitive processes impact attitudes,” Cognition and
Emotion, vol. 29, no. 1, pp. 1–26, 2015. [Online]. Available:
https://doi.org/10.1080/02699931.2014.967183
[24] A. Sawyer, “Repetition, cognitive response, and persuasion,” in
Cognitive Responses in Persuasion, R. Petty, T. Ostrom, and T. Brock,
Eds. Psychology Press, 1982, p. 26.
[25] H. Bless, F. Strack, and E. Walther, “Memory as a target of social
influence? memory distortions as a function of social influence
and metacognitive knowledge,” Social influence: Direct and indirect
processes, pp. 167–183, 2001.
[26] H. Kennedy and R. L. Hill, “The feeling of numbers: Emotions in
everyday engagements with data and their visualisation,” Sociol-
ogy, vol. 52, no. 4, pp. 830–848, 2018.
[27] S. Campbell and D. Offenhuber, “Feeling numbers: The emotional
impact of proximity techniques in visualization,” Information De-
sign Journal, vol. 25, no. 1, 2019.
[28] A. V. Pandey, A. Manivannan, O. Nov, M. Satterthwaite, and
E. Bertini, “The persuasive power of data visualization,” IEEE
Transactions on Visualization and Computer Graphics, vol. 20, no. 12,
pp. 2211–2220, 2014.
[29] M. A. Borkin, Z. Bylinskii, N. W. Kim, C. M. Bainbridge, C. S. Yeh,
D. Borkin, H. Pfister, and A. Oliva, “Beyond memorability: Visu-
alization recognition and recall,” IEEE Transactions on Visualization
and Computer Graphics, vol. 22, no. 1, pp. 519–528, 2015.
[30] H. O. Obie, C. Chua, I. Avazpour, M. Abdelrazek, J. Grundy, and
T. Bednarz, “A study of the effects of narration on comprehension
and memorability of visualisations,” Journal of Computer Languages,
vol. 52, pp. 113–124, 2019.
[31] M. A. Borkin, A. A. Vo, Z. Bylinskii, P. Isola, S. Sunkavalli, A. Oliva,
and H. Pfister, “What makes a visualization memorable?” IEEE
Transactions on Visualization and Computer Graphics, vol. 19, no. 12,
pp. 2306–2315, 2013.
[32] J. Matthes, “What’s in a frame? a content analysis of media
framing studies in the world’s leading communication journals,
1990-2005,” Journalism & Mass Communication Quarterly, vol. 86,
no. 2, pp. 349–367, 2009.
[33] S. K. Card, J. D. Mackinlay, and B. Shneiderman, “Information
visualization,” in Using Vision to Think. Morgan Kaufmann, 1999,
pp. 1–34.
[34] H.-K. Kong, Z. Liu, and K. Karahalios, “Frames and slants in titles
of visualizations on controversial topics,” in Proceedings of the 2018
CHI Conference on Human Factors in Computing Systems, 2018, pp.
1–12.
[35] H.-K. Kong, Z. Liu, and K. Karahalios, “Trust and recall of in-
formation across varying degrees of title-visualization misalign-
ment,” in Proceedings of the 2019 CHI Conference on Human Factors
in Computing Systems, 2019, pp. 1–13.
[36] E. R. Tufte, The visual display of quantitative information. Cheshire,
CT: Graphics Press, 1983.
[37] S. Haroz, R. Kosara, and S. L. Franconeri, “Isotype visualiza-
tion: Working memory, performance, and engagement with pic-
tographs,” in Proceedings of the 2015 CHI Conference on Human
Factors in Computing Systems, 2015, pp. 1191–1200.
[38] K. K. Reardon, Persuasion in practice. Sage, 1991.
[39] J. G. Jones and H. W. Simons, Persuasion in society. Routledge,
2017.
[40] P. Morseletto, “Analysing the influence of visualisations in
global environmental governance,” Environmental Science & Policy,
vol. 78, pp. 40–48, 2017.
[41] B. Schneider and T. Nocke, Image politics of climate change: visual-
izations, imaginations, documentations. transcript Verlag, 2014.
[42] L. van Beek, T. Metze, E. Kunseler, H. Huitzing, F. de Blois,
and A. Wardekker, “Environmental visualizations: Framing and
reframing between science, policy and society,” Environmental
Science & Policy, vol. 114, pp. 497–505, 2020.
[43] K. Rall, M. L. Satterthwaite, A. V. Pandey, J. Emerson, J. Boy,
O. Nov, and E. Bertini, “Data visualization for human rights
advocacy,” Journal of Human Rights Practice, vol. 8, no. 2, pp. 171–
197, 2016.
[44] M. Moya, “Persuasi
´
on y cambio de actitudes,” Psicolog´ıa Social, pp.
153–170, 1999.
[45] A. Iyer and J. Oldmeadow, “Picture this: Emotional and political
responses to photographs of the kenneth bigley kidnapping,”
European Journal of Social Psychology, vol. 36, no. 5, pp. 635–647,
2006.
[46] E. M. Peck, S. E. Ayuso, and O. El-Etr, “Data is personal: Attitudes
and perceptions of data visualization in rural pennsylvania,” in
Proceedings of the 2019 CHI Conference on Human Factors in Comput-
ing Systems, 2019, pp. 244:1–244:12.
[47] C. D’Ignazio and L. F. Klein, “Feminist data visualization,”
VIS4DH: 2016 Workshop on Visualization for the Digital Humanities,
2016.
[48] C. D’Ignazio and L. F. Klein, Data Feminism. MIT Press, 2020.
[49] D. Albarrac
´
ın, W. Wang, H. Li, and K. Noguchi, “Structure of
attitudes: Judgments, memory, and implications for change,” in
Attitudes and Attitude Change, W. D. Crano and R. Prislin, Eds.
Psychology Press, 2008.
[50] H. Ohira, W. M. Winton, and M. Oyama, “Effects of stimulus va-
lence on recognition memory and endogenous eyeblinks: Further
evidence for positive-negative asymmetry,” Personality and Social
Psychology Bulletin, vol. 24, no. 9, pp. 986–993, 1998.
[51] P. David, “News concreteness and visual-verbal association: Do
news pictures narrow the recall gap between concrete and abstract
news?” Human Communication Research, vol. 25, no. 2, pp. 180–201,
1998.
[52] S. Tindall-Ford, P. Chandler, and J. Sweller, “When two sensory
modes are better than one.” Journal of Experimental Psychology:
Applied, vol. 3, no. 4, p. 257–287, 1997.
[53] M. Garry, D. Strange, D. M. Bernstein, and T. Kinzett, “Pho-
tographs can distort memory for the news,” Applied Cognitive
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 16
Psychology: The Official Journal of the Society for Applied Research in
Memory and Cognition, vol. 21, no. 8, pp. 995–1004, 2007.
[54] Y.-S. Kim, L. A. Walls, P. Krafft, and J. Hullman, “A bayesian
cognition approach to improve data visualization,” in Proceedings
of the 2019 chi conference on human factors in computing systems, 2019,
pp. 1–14.
[55] Y.-S. Kim, K. Reinecke, and J. Hullman, “Data through others’
eyes: The impact of visualizing others’ expectations on visualiza-
tion interpretation,” IEEE Transactions on Visualization and Com-
puter Graphics, vol. 24, no. 1, pp. 760–769, 2018.
[56] Y.-S. Kim, P. Kayongo, M. Grunde-McLaughlin, and J. Hullman,
“Bayesian-assisted inference from visualized data,” IEEE Trans-
actions on Visualization and Computer Graphics, vol. 27, no. 2, pp.
989–999, 2021.
[57] The Guardian, “The new normal? how cli-
mate change is making droughts worse,” 2019,
https://www.theguardian.com/environment/ng-
interactive/2018/oct/03/the-new-normal-how-climate-change-
is-making-droughts-worse, Last accessed on 2023-01-10.
[58] New York Times, “A quarter of human-
ity faces looming water crises,” 2019,
https://www.nytimes.com/interactive/2019/08/06/climate/world-
water-stress.html , Last accessed on 2023-01-10.
[59] Reuters, “The race to save the river ganges,”
2019, https://www.reuters.com/graphics/INDIA-
RIVER/010081TW39P/index.html, Last accessed on 2023-01-10.
[60] Z. Wang, S. Wang, M. Farinella, D. Murray-Rust, N. Henry Riche,
and B. Bach, “Comparing effectiveness and engagement of data
comics and infographics,” in Proceedings of the 2019 CHI Conference
on Human Factors in Computing Systems, 2019, pp. 1–12.
[61] S. Jacobson, J. Marino, and R. E. Gutsche Jr, “The digital animation
of literary journalism,” Journalism, vol. 17, no. 4, pp. 527–546, 2016.
[62] D. Watson, L. A. Clark, and A. Tellegen, “Development and
validation of brief measures of positive and negative affect: the
panas scales.” Journal of Personality and Social Psychology, vol. 54,
no. 6, p. 1063, 1988.
[63] E. Thompson, “Development and validation of an internationally
reliable short-form of the positive and negative affect schedule
(panas), 38 (2), 227–242,” 2007.
[64] M. Dufey and A. M. Fernandez, “Validez y confiabilidad del
positive affect and negative affect schedule (panas) en estudiantes
universitarios chilenos,” Revista Iberoamericana de Diagn´ostico y
Evaluaci´on-e Avalia¸c˜ao Psicol´ogica, vol. 2, no. 34, pp. 157–173, 2012.
[65] K. Charmaz, “La teor
´
ıa fundamentada en el siglo xxi: Aplicaciones
para promover estudios sobre la justicia social,” in Manual de
investigaci´on cualitativa. Gedisa, 2013, pp. 270–325.
[66] S. M. Kelders, R. N. Kok, H. C. Ossebaard, and J. E. Van Gemert-
Pijnen, “Persuasive system design does matter: a systematic re-
view of adherence to web-based interventions,” Journal of medical
Internet research, vol. 14, no. 6, p. e2104, 2012.
[67] P. Dragicevic, “Fair statistical communication in hci,” in Modern
Statistical Methods for HCI. Springer, 2016, pp. 291–330.
[68] J. Liem, C. Perin, and J. Wood, “Structure and empathy in visual
data storytelling: Evaluating their influence on attitude,” in Com-
puter Graphics Forum, vol. 39, no. 3. Wiley Online Library, 2020,
pp. 277–289.
[69] V. Pe
˜
na-Araya, E. Pietriga, and A. Bezerianos, “A comparison of
visualizations for identifying correlation over space and time,”
IEEE Transactions on Visualization and Computer Graphics, vol. 26,
no. 1, pp. 375–385, 2019.
[70] T. Blascheck, L. Besanc¸on, A. Bezerianos, B. Lee, and P. Isenberg,
“Glanceable visualization: Studies of data comparison perfor-
mance on smartwatches,” IEEE Transactions on Visualization and
Computer Graphics, vol. 25, no. 1, pp. 630–640, 2019.
[71] L. Svicarovic, D. Parra, and M. J. Lobo, “Evaluating Interactive
Comparison Techniques in a Multiclass Density Map for Vi-
sual Crime Analytics,” in EuroVis 2021 - Short Papers, M. Agus,
C. Garth, and A. Kerren, Eds. The Eurographics Association,
2021.
[72] L. Besanc¸on and P. Dragicevic, “The Significant Difference be-
tween p-values and Confidence Intervals,” in 29`eme Conf´erence
Francophone sur l’Interaction Homme-Machine, AFIHM, Ed., 2017,
https://hal.inria.fr/hal-01562281.
[73] M. Krzywinski and N. Altman, “Error bars: the meaning of error
bars is often misinterpreted, as is the statistical significance of their
overlap,” Nature methods, vol. 10, no. 10, pp. 921–923, 2013.
[74] L. F. Barrett and J. A. Russell, “The structure of current affect:
Controversies and emerging consensus,” Current directions in psy-
chological science, vol. 8, no. 1, pp. 10–14, 1999.
Manuela Garret
´
on is an Assistant Professor in
the School of Design at Pontificia Universidad
Cat
´
olica de Chile. She received a master’s de-
gree in the Interactive Telecommunications Pro-
gramme at New York University, and she is a
PhD student in the Department of Computer
Science, also at Pontificia Universidad Cat
´
olica
de Chile. Her research interests include visual-
ization in the context of public policy and their
intersection with data journalism.
Francesca Morini is a PhD student in Media
and Communication Studies in the School of
Culture and Education at S
¨
odert
¨
orn University
in Sweden and a research associate at the Ur-
ban Futures Institute of the University of Applied
Sciences Potsdam. Her research interests lie in
exploring the intersection of information visual-
ization and data journalism.
Pablo Celhay is an Assistant Professor at the
Government School and Department of Eco-
nomics at the Pontificia Universidad Cat
´
olica de
Chile. His research includes health economics,
development economics and impact evaluation
methods. He has been a consultant and advi-
sor for, the Inter-American Development Bank,
World Bank, UNICEF, and UNDP. He received
his PhD from the Harris School at the University
of Chicago.
Marian D
¨
ork is a research professor for infor-
mation visualization at the Department of Design
and Institute for Urban Futures of the University
of Applied Sciences Potsdam. As a doctoral stu-
dent at the University of Calgary (2008-2012)
and a postdoc at Newcastle University (2012-
2013) he designed and studied interactive visu-
alizations to support information seeking. Since
2015 he has co-directed the UCLAB, a trans-
disciplinary research space at the intersection
between computing, design, and the humanities.
Denis Parra is Associate Professor in the De-
partment of Computer Science at Pontificia Uni-
versidad Cat
´
olicad de Chile. He is researcher
at excellence research centers in Chile CENIA,
iHealth and IMFD. He received a professional
title of Civil Engineer in Informatics in 2004 from
UACh, and a Ph.D. in Information Science from
the University of Pittsburgh, USA. His research
interests are Recommender Systems, Intelligent
User Interfaces, Applications of Machine Learn-
ing and Information Visualization.
This article has been accepted for publication in IEEE Transactions on Visualization and Computer Graphics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVCG.2023.3248319
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/